Organization on High : Expanding Use of Physician Examination Scores

As pressure mounts for physicians to take more frequent, costly examinations, some studies purport to show that examination scores correlate with performance measures: indices of preventive care and disease management in primary care, and complaints to licensure boards. The idea that physicians will be better at their tasks in proportion to the amount of knowledge they possess is unassailable on its face validity alone. The idea that government licensure examinations can measure this knowledge and use it to predict important practices of physicians is, however, ill founded. Examinations can predict some things, but not very important things. The data-driven approach, moreover, has great potential for misuse by those at nodes of central political and economic power. Political control, not science, appears to be the real agenda served by such studies. In 2002, the American Medical Association (AMA) published a study, done in Canada, showing that scores on licensure examinations correlate with six measures supposed to represent physician competency. In 2007, another study done by many of the same authors showed that scores on a national skills examination predicted complaints to medical regulatory authorities. In defense of such studies, it may be said that perfection is not to be expected, that competence is important, that incompetence abounds, and that the states have to do something to protect the public from medical harm. Medical educators need feedback in order to improve their methods. Nothing in the studies will be misused by those in positions of authority. Their methodology, however, has a magnetic attraction for centrists. It is an aviator’s view, taking in a broad scope of landscape without the bother of fine detail. The practice of medicine needs broad epidemiologic views, but the actual delivery of care must be done with the feet on the ground, face to face with patients. The latter perspective has to rule in the end. The intention here is to offer a corrective, lest anyone in some organization on high try to misappropriate the findings of Dr. Robyn Tamblyn and her associates. necessarily ACanadian study, published in 2002 by theAMA, showed that licensure examination scores correlated with six measures of physician competency. A later study by many of the same authors, published in 2007, showed that scores on a national skills examinations correlated with complaints to regulatory authorities. Articles such as these are likely to be cited as justifications for subjecting physicians to periodic, costly examinations as a condition ofmaintaining licensure. It is tempting to join the argument at the level of data—answering a salvo of their favorite studies with a salvo of ours. Informational depth charges answer a spread of informational torpedoes. Sun Tzu, in his ancient classic said, “The 1